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Abstract 

In this paper, we formulate and solve a randomized optimal consensus problem for multi- 
agent systems with stochastically time-varying interconnection topology. The considered 
multi-agent system with a simple randomized iterating rule achieves an almost sure consen- 
sus meanwhile solving the optimization problem mm zend J27=i m which the optimal 
solution set of objective function fa can only be observed by agent i itself. At each time step, 
simply determined by a Bernoulli trial, each agent independently and randomly chooses ei- 
ther taking an average among its neighbor set, or projecting onto the optimal solution set 
of its own optimization component. Both directed and bidirectional communication graphs 
are studied. Connectivity conditions are proposed to guarantee an optimal consensus al- 
most surely with proper convexity and intersection assumptions. The convergence analysis 
is carried out using convex analysis. We compare the randomized algorithm with the de- 
terministic one via a numerical example. The results illustrate that a group of autonomous 
agents can reach an optimal opinion by each node simply making a randomized trade-off 
between following its neighbors or sticking to its own opinion at each time step. 

Keywords: Multi-agent systems, Optimal consensus, Set convergence, Distributed opti- 
mization, Randomized algorithms 

1 Introduction 

In recent years, there have been considerable research efforts on multi-agent dynamics in ap- 
plication areas such as engineering, natural science, and social science. Cooperative control of 
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multi-agent systems is an active research topic, where collective tasks are enabled by the recent 
developments of distributed control protocols via interconnected communication [6j [7J EJ El HU1 
[TT | ll2 | [T3 l ll5 1 [T6 l l20j. However, fundamental difficulties remain in the search of suitable tools to 
describe and design the dynamical behavior of these systems and thus to provide insights in their 
basic principles. Unlike what is often the case in classical control design, multi- agent control 
systems aim at fully exploiting, rather than attenuating, the interconnection between subsys- 
tems. The distributed nature of the information processing and control requires completely new 
approaches to analysis and synthesis. 

Consensus is a central problem in the study of multi-agent systems, which usually requires 
that all the agents achieve the same state, e.g., a certain relative position or velocity. Efforts 
have been devoted to characterize the fundamental link between agent dynamics and group 
coordination, in which the connectivity of the multi-agent network plays a key role. Switching 
topologies in different cases, and the "joint connection" or similar concepts are important in the 
analysis of stability and convergence. Uniform joint-connection, i.e., the joint graph is connected 
during all intervals which are longer than a constant, has been employed for various consensus 
problems [U [JJ [22J [T7] . On the other hand, [t, oo)-joint connectedness, i.e., the joint graph is 
connected in time intervals [i,oo), is the most general form to secure the global coordination, 
which is also proved to be necessary in many situations [SI [TH]. Moreover, consensus seeking 
over randomly varying networks has been proposed in the literature [24 1 I25[ [26l \27\ 128] . in which 
the communication graph is usually modeled a sequence of i.i.d. random variables over time. 

Minimizing a sum of functions, Y^i=i fi( z )i using distributed algorithms, where each com- 
ponent function /j is known only to a particular agent i, has attracted much attention in recent 
years, due to its wide application in multi-agent systems and resource allocation in wireless 
networks [29[ l32l 1301 [3T| [Ml 135] . A class of subgradient-based incremental algorithms when 
some estimate of the optimal solution can be passed over the network via deterministic or ran- 
domized iteration, were studied in |29[ [321 142] . Then in [Mj a non-gradient-based algorithm 
was proposed, where each node starts at its own optimal solution and updates using a pairwise 
equalizing protocol. The local information transmitted over the neighborhood is usually limited 
to a convex combination of its neighbors [3 [JJ [8] . Combing the ideas of consensus algorithms 
and subgradient methods has resulted in a number of significant results. A subgradient method 
in combination with consensus steps was given for solving coupled optimization problems with 
fixed undirected topology in [31J. An important contribution on multi-agent optimization is 
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|40j , in which the presented decentralized algorithm was based on simply summing an averaging 
(consensus) part and a subgradient part, and convergence bounds for a distributed multi-agent 
computation model with time- varying communication graphs with various connectivity assump- 
tions were shown. A constrained optimization problem was studied in [41] , where each agent is 
assumed to always lie in a particular convex set, and consensus and optimization were shown to 
be guaranteed together by each agent taking projection onto its own set at each step. Augmented 
lagrangian algorithms with directed gossip communication to solve the constrained optimization 
problem in [33]. Then a convex-projection-based distributed control was presented for multi- 
agent systems with continuous-time dynamics to solve this optimization problem asymptotically 

In this paper, we present a randomized multi-agent optimization algorithm. Different from 
the existing results, we focus on the randomization of individual decision-making of each node. 
We assume that each optimal solution set of is a closed convex set, and can be observed only 
by node i. Assuming that the intersection of all the solution sets is nonempty, the optimal solu- 
tion set of the group objective becomes this intersection se. Then the optimization problem is 
equivalent to a distributed intersection computation problem. Computing convex sets' intersec- 
tion is actually a classical problem. Alternating projection algorithm was a standard centralized 
solution, which was discussed in [371 EH1 [Ml HI]- Then the projected consensus algorithm was 
presented in |41j . 

We propose a randomized algorithm as follows. At each time step, there are two options 
for each agent: a standard averaging (consensus) part as a convex combination of its neighbors' 
state, and a projection part as the convex projection of its current state onto its own optimal 
solution set. In the algorithm, each agent independently makes a decision via a simple Bernoulli 
trial, i.e., chooses the averaging part with probability p, and the projection part with probability 
1 — p. This algorithm is a randomized version of the projected consensus algorithm in |41j . 
Viewing the state of each agent as its "opinion", one can interpret the randomized algorithm 
considered in this paper as a model of spread of information in social networks [28J. In this case, 
the averaging part of the iteration corresponds to an agent updating its opinion based on its 
neighbors' information, while the projection part corresponds to an agent updating its opinion 
based only on its own belief of what is the best move. The authors of [28J draw interesting 
conclusions from a model similar to ours on how misinformation can spread in a social network. 

In our model, the communication graph is assumed to be a general random digraph pro- 



3 



cess independent with the agents' decision making process. Instead of assuming that the com- 
munication graph is modeled by a sequence of i.i.d. random variables over time, we just re- 
quire the connectivity-independence condition, which is essentially different with existing works 
\25 \ \27 \ [26] . Borrowing the ideas on uniform joint-connection [6j[71[22] and [t, oo)-joint connect- 
edness [H [18], we introduce connectivity conditions of stochastically uniformly (jointly) strongly 
connected (SUSC) and stochastically infinitely (jointly) connected (SIC) graphs, respectively. 
The results show that the considered multi-agent network can almost surely achieve a global 
optimal consensus, i.e., a global consensus within the optimal solution set of Y^i=\ fi( z )i when 
the communication graph is SUSC with general directed graphs, or SIC with bidirectional in- 
formation exchange. Convergence is derived with the help of convex analysis and probabilistic 
analysis. 

The paper is organized as follows. In Section 2, some preliminary concepts are introduced. 
In Section 3, we formulate the considered multi-agent optimization model and present the op- 
timization algorithm. We also establish some basic assumptions and lemmas in this section. 
Then the main result and convergence analysis are shown for directed and bidirectional graphs, 
respectively in Sections 4 and 5. In Section 6 we study a numerical example. Finally, concluding 
remarks are given in Section 7. 

2 Preliminaries 

Here we introduce some mathematical notations and tools on graph theory [5], convex analysis 
PIE] and Bernoulli trials [I]. 

2.1 Directed Graphs 

A directed graph (digraph) Q = (V,£) consists of a finite set V = {1, . . . , n} of nodes and an 
arc set £ . An element e = (i,j) £ £, which is an ordered pair of nodes i,j £ V, is called an 
arc leaving from node i and entering node j. If the e^'s are pairwise distinct in an alternating 
sequence voe\v\e2V2 ■ ■ ■ e n v n of nodes Vi and arcs ej = (vi-i,Vi) £ £ for i = 1,2, ... ,n, the 
sequence is called a (directed) path. A path from i to j is denoted i — > j. Q is said to be strongly 
connected if it contains paths i —¥ j and j — > i for every pair of nodes i and j. 

A weighted digraph Q is a digraph with weights assigned for its arcs. A weighted digraph Q 
is called to be bidirectional if for any two nodes i and j, (i,j) £ £ if and only if (j, i) £ £ , but 
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the weights of and may be different. A bidirectional digraph is strongly connected if 
and only if it is connected as an undirected graph (ignoring the directions of the arcs). 

The adjacency matrix, A, of digraph Q is the n x n matrix whose ij-entry, A«, is 1 if there 
is an arc from i to j, and otherwise. Additionally if Q\ = (V, £\) and Q2 = (V, £2) have the 
same node set, the union of the two digraphs is defined as G1UQ2 = (V, £\ U £2)- 

2.2 Convex Analysis 

A set K C R d (d > 0) is said to be convex if (1 — \)x + \y G K whenever x,y G K and < A < 1. 
For any set S C M> d , the intersection of all convex sets containing S is called the convex hull of 
S, and is denoted by co(S). 

Let K be a closed convex set in and denote \x\k — inf^gft- |x — y\ as the distance between 
x G and fT, where | • | denotes the Euclidean norm. Then we can associate to any x G R d 
a unique element Pk{x) G X satisfying |x — Pk{x)\ = \x\k, where the map Pk is called the 
projector onto K with 

(P K (x) - x, P K (x) -y)<0, Vy G K. (1) 
Moreover, we have the following non-expansiveness property for Px- 

\P K (x) - P K (y)\ < \x-y\, x,y£R d . (2) 

A function / : H d — > 1R is said to be convex if it satisfies 

f(av + (1 - a)w) < af(v) + (1 - a)f(w), (3) 

for all v,w G and < a < 1. The following lemma holds (Example 3.16, pp. 88, [2]). 

Lemma 2.1 Lei K be a convex set in H d . Then \x\k is a convex function. 

The next lemma can be found in [I]. 

Lemma 2.2 Let K be a subset ofM, d . The convex hull co(K) of K is the set of elements of the 
form 

d+l 

X — ^ ^ ^iXi, 
i=l 

where Aj > 0, i = 1, . . . , d + 1 with Yli=i -^i = 1 an d Xi G K . 

Additionally, for every two vectors ^ ^1,^2 G we define their angle as 4>{v\,V2) G [0,7r] 
with coscf) = {vi, f2)/|^i| • |f 2I - 
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2.3 Bernoulli Trials 

A Bernoulli trial is a binary random variable which only takes two values and 1. Let 
Yi,Y2,Yz, . . . be a sequence of independent Bernoulli trials such that for each k = 1,2,..., 
the probability that = 1 is p\. € [0, 1]. Here pk is called the success probability for Yj,. 
Then the next lemma holds. The proof is obvious, and therefore omitted. 

Lemma 2.3 Let k = 1, 2, ... , be a sequence of independent Bernoulli trials, where the suc- 
cess probability ofYk is pk £ [0, 1]. Suppose there exists a constant p* > such that pk > p* for 
all k. Then we have P(Vfe = 1 for infinitely many k > l) = 1. 

3 Problem Formulation 

In this section, we formulate the considered optimal consensus problem. We propose a multi- 
agent optimization model, and then introduce a neighbor-based randomized optimization algo- 
rithm. We also introduce key assumptions and establish two basic lemmas on the algorithm 
used in the subsequent analysis. 

3.1 Multi-agent Model 

Consider a multi- agent system with agent set V = {1, 2, . . . , n}. The objective of the network is 
to reach a consensus, and meanwhile to cooperatively solve the following optimization problem 



where fi : R — >■ R represents the cost function of agent i, observed by agent i only, and z is a 
decision vector. 

Time is slotted, and the dynamics of the network is in discrete time. Each agent i starts with 
an arbitrary initial position, denoted Xj(0) € R d , and updates its state Xj(fc) for k = 0, 1, 2, ... , 
based on the information received from its neighbors and the information observed from its 
optimization component /j. 

3.1.1 Communication Graph 

We suppose the communication graph over the multi-agent network is a stochastic digraph 
process Gk = (V,£fc), k = 0, 1, To be precise, the ij-entry A{j(k) of the adjacency matrix, 



n 



mm 




(4) 



i=l 
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A{k) of Qk, is a general {0, l}-state stochastic process. We assume there is no self-looped arc in 
the communication graphs, i.e., Au(k) = for all i and k. We use the following assumption on 
the independence of Gk- 

Al ( Connectivity Independence) Events Ck = {Gk is connected (in certain sense)}, k = 0, 1, ... , 
are independent. 

Remark 3.1 Connectivity independence means that a sequence of random variables w{k), which 
are defined by that w{k) = 1 if Gk is connected (in certain sense) and w{k) = otherwise, 
are independent. Note that, different with existing works }25[ \27[ \26$ . we do not impose the 
assumption that w(k), k = 0, . . . , are identically distributed. 

At time k, node j is said to be a neighbor of i if there is an arc £ £fc. Particularly, we 
assume that each node is always a neighbor of itself. Let A/i(/c) represent the set of agent i's 
neighbors at time k. 

Denote the joint graph of Gk m time interval [k\, k<z\ as G([ki, A^]) = (V, U tg [fc 1 ^S^t)), where 
< k\ < k2 < +oo. Then we have the following definition. 

Definition 3.1 (i) Gk is said to be stochastically uniformly (jointly) strongly connected (SUSC) 
if there exist two constants B > 1 and < q < 1 such that for any k > 0, 



(ii) Assume that Gk is bidirectional for all k > 0. Then Gk is said to be stochastically 
infinitely (jointly) connected (SIC) if there exist a (deterministic) sequence = &q < • • • < k* < 
k* +l < . . . and a constant < q < 1 such that for all r = 0,1, ... , 



3.1.2 Neighboring Information 

The local information that each agent uses to update its state consists of two parts: the averaging 
and the projection parts. The averaging part is defined as 



where dij{k) > 0, i, j = 1, . . . , n are the arc weights. The weights fulfill the following assumption: 
A2 (Arc Weights) (i) a ij{k) = 1 for all % and k. 




P ( G{[k*, is connected) > q. 
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(ii) There exists a constant n > such that r] < £%(A;) for all i, j and k. 
The projection part is defined as 

9i (k) = P Xi ( Xi (k)), 

where Xi = {v \ fi(v) = min 2gIEl d fi(z)} is the optimal solution set of each objective function 

fi, i = 1, . . . , n. We use the following assumptions. 

A3 ( Convex Solution Set) Xi,i = 1, . . . , n, are closed convex sets. 

n 

A 4 (Nonempty Intersection) Xq = f] Xi is nonempty. 

i=i 

In the rest of the paper, A1-A4 are our standing assumptions. 

Remark 3.2 The average ei(k) has been widely used in consensus algorithms, e.g., |6l [7| \Bjj- 
Assumption A2(i) indicates that ei(k) is always within the convex hull of node i's neighbors, 
i.e., co{xj(k), j £ Mi(k)}, and, moreover, A2(ii) ensures that ei(k) is in the relative interior of 
co{ Xj (k),j eAfi(k)} [2i 



Remark 3.3 As Xi can be observed by node i, Pxi{xi{k)) can be easily obtained. Note that, for 
a convex set K C H d , we have that V\z\ 2 K = 2(Pk(z) — z) [1]. Therefore, for instance, in order 
to compute Px^Xi^k)), node i may first establish a local coordinate system, and then construct 
a function h(z) = \z\ x /2 to compute Vh(xi(k)) within this coordinate system. Then we know 
PxMiik)) = xi{k) + Vh(xi(k)). 

3.1.3 Randomized Algorithm 

We are now ready to introduce the randomized optimization algorithm. At each time step, each 
agent independently and randomly either takes an average among its time- varying neighbor set, 
or projects onto the optimal solution set of its own objective function: 



Xi(k + l) 



J2jeN (k) a.ij{k)xj(k), with probability p 

(5) 



Px l (xi(k)), with probability 1 — p 

where < p < 1 is a given constant. 

Remark 3.4 One motivation for the study of algorithm |5p follows from the literature on opin- 
ion dynamics in social networks, where each agent makes a choice randomly between sticking to 
its own observation and following its neighbors' opinion fUfflj . An interesting question is whether 
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Figure 1: The goal of the multi-agent network is to achieve a consensus in the optimal solution 
set Xq. 

the social network reaches a common opinion or not, and if the answer is yes, whether the 
network could reach an optimal common opinion. 

On the other hand, from an engineering viewpoint, different from most existing works 1361 
\41\ \32jl . the randomized algorithm |5p gives freedom to the nodes to choose to compute (projec- 
tion), or communicate (averaging) independently with others at each time k. This provides an 
important tradeoff between control, computation and communication as in algorithm |5p, each 
node is not synchronously required to both compute and communicate in each time step. 

Remark 3.5 The constrained consensus algorithm studied in [J^l], can be viewed as a determin- 
istic case of in which each node alternate between averaging and projection in the iterations. 

With assumptions A3 and A4, Xq becomes the global optimal solution set of Yli=i fi( z )- 
Let {x(k;x°) = (xf(k;x ),...,x^(k;x°)) T ^ c ^ =0 be the stochastic sequence generated by ^ 
with initial condition x ° = (x\ (0), . . . , xl(0)) T e R nd . We will identify x(k; x°) with x(k) where 
there is no possible confusion. The considered optimal consensus problem is defined as follows. 
See Figure [I] for an illustration. 

Definition 3.2 (i) A global optimal set aggregation is achieved almost surely (a.s.) for algo- 
rithm @ if for all x° € R nd , 

P( lim \ Xi (k)\ Xo =0,i = l,...,n) =1. (6) 
9 



(ii) A global consensus is achieved almost surely (a.s.) for algorithm ^ if for all 

P( lim \ Xi (k) -xj(k)\ =0,i,j = l,...,n) = 1. (7) 

\ k— >+oo / 

(Hi) A global optimal consensus is achieved almost surely (a.s.) for algorithm pp if both |6p 
and |?p hold. 

3.2 Basic Properties 

In this subsection, we establish two key lemmas on the algorithm ([5]). 

Lemma 3.1 Let K be a closed convex set in m d , and Kq C K be a convex subset of K. Then 
for any y G K, d , we have 

\PK(y)\ 2 Ko + \y\ 2 K <\y\ 

Proof. According to Q, we know that 

(P K (y)-y,P K (y)-P Ko (y)} < 0. 

Therefore, we obtain 

(P K (y) -y,y- P Ko {y)) = (P K (y) -y,y- P K {y) + P K {y) - P Ko {y)) < -\v\k- 

Then, 

\P K (y)\ 2 K = \PK(y)-PK (PK(y))\ 2 
<\P K (y)-PK (y)\ 2 
= \P K (y)-y + y-P Ko (y)\ 2 
= \y\ 2 K + \v\k + *{Pk(v) -y,y- PrM) 

I |2 I 1 2 

< \y\K - \y\K- 

The desired conclusion follows. □ 

Lemma 3.2 Let {x(k) = (xf(k), . . . ,x^(k)) T }^L be a stochastic sequence defined by (^). Then 
for all k > and along every possible sample path, we have 

max \xi(k + l)\ Xo < max \xi(k)\x - 
i=l, ...,n i=l,. ...n 
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Proof. Take / G V. If node I takes averaging at time k, we have 

\xi(k + l)\ Xo = \P Xl {xi(k))\ Xo = \P Xl { Xl {k)) - P Xo (P X[ (xi(k)))\ 

<|Px,(a:i(*:))-i^o(^(*))l 

< \ Xl {k) - P Xo (xi(k))\ 

< max \xi(k)\ Xo . (8) 
i=l,...,n 



On the other hand, if node i takes projection at time k, according to Lemma 2.1, we have 



X 



\xi(k + i)\ Xo = \ ^2 a ij( k ) x j( k ) 

< a lj( k )\ x j( k )\Xo 

< max \xi(k)\ Xo . (9) 

i=l,. ..,n 

Hence, the conclusion holds. □ 



Based on Lemma 3.2 we know that the following limit exists: 



£ = lim max \xi(k)\ Xo . 

k— ¥00 8=1,. ..,11 

It is immediate that the global optimal set aggregation is achieved almost surely if and only if 
P{£ = 0} = 1. 

Algorithm ^ is nonlinear and stochastic, and therefore quite challenging to analyze. As will 
be shown in the following, the communication graph plays an essential role on the convergence 
of the algorithm. In particular, directed and bidirectional graphs lead to different conditions for 
consensus. Hence, in the following two sections, we consider these two cases separately. 

4 Directed Graphs 

In this section, we give a connectivity condition guaranteeing an almost surely global optimal 
consensus for directed communication graphs. 
The main result is stated as follows. 

Theorem 4.1 Algorithm |5p achieves a global optimal consensus a.s. if Qk is SUSC. 



In order to prove Theorem 4.1, on one hand, we have to prove that all the agents converge 
to the global optimal solution set, i.e., Xq; and on the other hand that consensus is achieved. 
The proof divided into these two parts is given in the following two subsections. 
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4.1 Set Convergence 

In this subsection, we present the optimal set aggregation analysis of ([5]). Define 

Si =lim sup \xi(k)\xi, i = l,...,n. 

k— >oo 

Let A = {£ > 0} and M. = {3io s.t. 5i > 0} be two events, indicating that convergence to 
Xq for all the agents fails and convergence to Xi fails for some node io, respectively. The next 
lemma shows the relation between the two events. 

Lemma 4.1 P(AnM)=0ifG k is SUSC. 

Proof. Let {x^ik)}^^ ^ e a sam pl e sequence. Take an arbitrary node io G V. Then there exists 
a time sequence k\{uj) < • • • < k m {ui) < . . . with lim m _j.oo k m (u) = oo such that 

K(k m (cj))\ Xio >^ M>o. (io) 



Moreover, according to Lemma 3.2, W = 1, 2, . . . , 3T(£,oj) > such that 



k>T =► < K(fc)|jr < CM + i i = l,...,n. (11) 

In the following, k m {u) and T(^, uj) will be denoted as k m and T to simplify the notations. Note 
that they are both random variables. We divide the rest of the proof into three steps. 
Step 1. Suppose m is sufficiently large so that k m > T. We give an upper bound to node io in 
this step. 

Since node io projects onto Xi with probability 1 — p, Lemma |3.1| implies 



?(\x l0 {k m + l)\ Xo <^(i + ^Y- ) > 1 - p. (12) 
At time k m + 2, either one of two cases can happen in the update. 
• If node io chooses the projection option at time k m + 1, we have 

\x i0 (k m + 2)\xo = \x i0 (k m + l)\ Xo < ^^ + ^)2-lsl. ( 13 ) 



• If node io chooses the averaging option at time k m + 1, with ( 11 ), we can obtain from the 
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weights rule and Lemma 2.1 that 



\ x io 

{k m + 2)\ Xo 



^ a ioj{k m + l)xj(k m + 1) 

jeAfi (k m +l) 



< a ioio (k m + l)|x» (A; m + l)\ Xo + (1 



1 J + 1 > \/ (£ + \? ~ \sf + (1 - a iolo (k m + !))(£ + - t 



(14) 



Both (13) and (14) lead to 



p(|x i0 (A; m + 2)\ Xo < V ]j(Z+ \) 2 ~ \sf Q + (1 - t,){£ + -)) > 1 -p. (15) 
Continuing similar analysis, we further obtain 



P(K(*m + ^)U < rf~ X y (C + ^) 2 - i«5? + (1 - 7f" + 1), r = 1, 2, . . . ) > 1 - p. (16) 
Step 2. In this step, we continue to bound another node. Since Qk is SUSC, we have 
P^7([&m + 1) k m + £?]) is strongly connected) > q. 

which implies 

p(b k\ £ [k m + l,km + B] and i\ eV.ii/ io s.t. {io,h) G ^) > g. 
Let k\ = h m + g with 1 < g < B. Noting the fact that 



(fc m + £))(£ + 7 ) 



and based on (16), we have 



P ( \->n (>,„ + II v„ < /r \/ (* + ^) 2 - \sl + (1 - r/<0(£ + j)\T 



> P(ii chooses averaging at time A: m + g 
= P 



(17) 



where J-q = {io chooses projection at time fc m }. Therefore, with (16) and (17), we obtain 



P(3i a / io s.t. \x H (km + B + r)\ Xo < rj^^ + \f - ^? + (1 - t^" 1 )^ + i), 
Z = 0,l;r= 1,2,...) > (1 - p)pq- 



13 



Step 3. Repeating the analysis on time interval [k m + B + l,k m + 2B], there exists a node 
*2 ^ {*0i ^1} such that there is an arc leaving from {io, i\} entering ii in Q{[k m + B + 1, /c m + 21?]) 
with probability at least q. The estimate of \xi 2 (k m + 21? + r)|x is therefore can be similarly 
obtained. 

The upper analysis process can be carried out continuingly on intervals [k m + 21? + 1, k m + 
31?], . . . , [k m +(n—2)B+l, k m +(n— 1)B], and 13, ... , i n -\ can be found until V = {io, ii,..., i n -i}- 
Then one can obtain that for any i E V, 

P(|«i(* m + (n - 1)1? + 1)| X() < ^(-l)B^ + l)2_ lj2 + (1 _ ^(n-l)B )(€ + ^ eV ) 

= P ( max n Mfc m + (n - 1)1? + l)|x < ^ B ^ + \y - + (1 - i/"" 1 ^* + J)) 
> (1 -rfp™" 1 ^ 1 . (18) 

Moreover, we see from the previous analysis that the events 

Z m = { . max^ \ Xi {k m + {n- 1)1? + l)\ Xo < r,^ B ^ + \? - \% + (1 - r/^ 1 ^)^ + ])} 

are fully determined by the communication graph process and the node-decision process for all 
m with k m > T. Therefore, they can be viewed as a sequence of independent Bernoulli trials. 
Then based on Lemma |2.3| we see that with probability one, there is an infinite subsequence 
{kj, j = 1, 2, . . . } from {k m + (n — 1)1? + 1, k m > T} satisfying 

. max n \ Xl {k 3 )\ Xo < r? ( "- 1)B ^ + ^) 2 -^ o + (1 _ t,W)(£ + \). 

This implies 

P(K<) =1 (19) 

for all i = 1,2, ... , where K t = {i< 4 n ~ x)B {t + ~ |<%+(l-77 (n ~ 1)S )(£+i)}- As a result, 
we obtain P(ft*) = 1, where K* = lim^oc ^ = {£ < n {n ~ 1)B Jl 2 ~ zK + ( X _ ?7 ( ™~ 1)B )£}- 

Finally, it is not hard to see that A DM QK% because < ^ n ^ B < 1. The desired 
conclusion follows straightforwardly. □ 

Take a node ao £ V. Then define 

z ao (k) = max \xi(k)\ Xan - 

i=l,...,n u 

We also need the following fact to prove the optimal set convergence. 
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Lemma 4.2 Along every possible sample path of algorithm |5p and for all k, we have 

z ao (k + 1) < z ao (k) + max \xi(k)\ Xi . 

1=1,. ..,71 

Proof. For any node I = 1, . . . , n, if Z chooses the averaging part at time fc, we know that 

\xi(k + l)\ x =\ V aij(k)xj(k) < max \xi(k)\ x = z ao (k). (20) 

U I ' ' X an 4=1,. ..,71 U 

Moreover, if I chooses the projection part at time k, we have 

\x[(k + 1) - xi{k)\ = \xi{k)\ Xv 

which yields 

\xi(k + l)\ x < \xi(k)\ Xao + \xi(k)\ Xl < z ao (k) + max \xi(k)\ x . (21) 

u u 7 = 1,. ..,71 

according to the non-expansiveness property Q. Then the conclusion holds with (20) and (21). 

□ 



We are now in a place to present the optimal set convergence part of Theorem 4.1 as stated 
in the following conclusion. 

Proposition 4.1 Algorithm |5p achieves a global optimal set aggregation a.s. if Qk is SUSC. 
Proof. Note that, we have 

P(A) = P(AnM) +P(A(~)M C ) < P(A(~)M) +P(A\M C ). 



Since the conclusion is equivalent to P(»4.) = 0, with Lemma 4.1, we only need to prove 
P(A\M C ) = 0. 

Let {x u) (k)}^L Q be a sample sequence in M c . Then W = 1, 2, . . . , 3Ti(£, lj) > such that 

k>T x \xf(k)\ Xt < ], i = !,...,«. (22) 



Take an arbitrary node «o £ V. Based on Lemma 4.2 , we also have that for any {x w (k)}^L 6 
M c and s > T 1} 

z% (s + t)<z% (s) + j, r = 0,l,.... (23) 

We divide the rest part of the proof into three steps. 
Step 1. Denote k\ =T\. Since Qy. is SUSC, we have 

P^there exist k\ G [hi, hi + B — 1] and a\ 6 V s.i. (ao,ai) € £7^ J > 
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Let k\ = k\ + g, < g < B — 1. Then we obtain from the definition of (|5j) that 
P(Wi(&i + £> + l)|x Q0 < aaiao(h + g)\x ao (h + £>)|x Q0 + (1 - a aiao )z ao (h + £>)J > pq. (24) 



Thus, based on the weights rule Al and (22), (24) leads to 



P(W(fcl + g + l)|x ao < r/ • - + (1 - v)Mh) + Q ■ j)\M c ) > pq- 
Next, there will be two cases. 

• If node a± chooses the projection option at time k\ + g + 1, we have 

\x ai (h + g + 2)\ XaQ < \x ai (h + g + l)\ Xao + j 

<Vj + (l-V)(Za (h) + Q-j) + j. 

• If node a i chooses the averaging option at time k\ + g + 1, we have 

\x ai (h + Q + 2)| Xan < v\x ai {k\ + Q + l)|x an + (1 - v)za (h + Q + 1) 



(25) 



(26) 



< ?7b? • I + (! - ^)(^o(^i) + e ■ j)\ + (i - ^(^o(^i) + (<? + 1) ■ |) 



i 



i, 



<^-^ + (i-^)(« ao (fci) + (e + i)-^). 



(27) 



With (26) and (27), we obtain 



1, 1 



P (|x Ql (fci + g + 2) \ Xao < if ■ j + (1 - r? 2 ) (z ao ( kl ) + (g+l)-- e ) + - e \M c )>pq. (28 ) 
Then similar analysis yields that 

p(\ Xai{kl + g+T) \ Xao 4+(l-fl(W*l) + i± ^)+E<I 1 " 1 -). r = l,2,...|^ c ) > 



Furthermore, since < £ < -B — 1 and based on (22), it turns out that 

. n B+T JD I * _ 1 1 1 

p(|^(fc 1+jB + f)U ao < !L r + (l-r ? B+f )(^ (fci)+ ' ' 



: > + r 



f = 0,1,...; / = 0,l|.M c >pg. 



.S^ep 2. We continue the analysis on time interval [k\ + B,k\ + 2i? — 1]. There exists a node 
"2 {"0) a i} such that there is an arc leaving from {qo, «i} entering «2 inQ([ki+B, k m +2B— 1]) 
with probability Similarly we can obtain that for any f = 0, 1, . . . , 



P( \x a[ (h + 2B + f )| v < r? 2B +^ • - + (1 - 77^ +r )(z Qo (fcx) + (2B + f - 1) • 9) + 



1 



2BH 



1. 



1 



l = 0,l,2\M c ) >p z q. 



2„2 



£' 1 — 7? 
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We repeat the upper process on time intervals [k\ + 2B, k\ + 3B — 1] , . . . , [k m + (n — 2)B, k\ + 
(n — 1)B — 1], and 03, . . . , a n -\ can be found until V = {ao, a\, . . . , a. n -\}. Then one can obtain 
that 

P^Xiih + in-^Blx^ < (l-^-^z^^ + L-j, i = l,...,n\M c ) > V"\ 

where L = r/^" - " 1 ^ + (n — 1)[B + jz^]- Denote &2 = fci + (n — 1)B. Then we have 

1 



(ki) + L ■ j\M c j > p, 
where < 6> = 1 - r]^' 1 ^ < 1 and < p = p*" 1 ^ 1 < 1. 

5*iep 5. Let k m = k\ + (m — l)(n — 1)B, m = 3,4, . . . . Based on similar analysis, we see that 

1 



V[z ao {k m+1 ) < d z ao (k m ) + L ■ j\M c j >p, m = 3,4, .... 

Then we can define a random variable x independently with z ao (k m ),m = 1, . . . , such that 

1, with probability 1 — p 
60 , with probability p 



X 



(29) 



As a result, with (23) and (29), we conclude that for any m = 1, 2, . . . , 



which implies 



p(z ao (k m+ i) < x- z ao (k m ) +L - j\M c ^j = 1, 
E(z ao (k rn+1 )\M c ) < (1 - (1 - e Q )p)v{z ao (k m )\M c ) +L-j. 



Therefore, we can further obtain 



limsupE(2: ao (fc m )|.A/f c ) < 



1 



(1-0 O )P I 



(30) 



Since I can be any positive integer in (30) and z ao (k m ) is nonnegative for any m, we have 

lim B(z ao (k m )\M c ) =0. (31) 
Based on Fatou's lemma, we know 

0<E( lim z ao (k m )\M c ) < lim B(z ao {k m )\M c ) =0, (32) 

which yields 

P( lim z ao (k m ) = 0\M C ) = 1. (33) 



Finally, because ao is chosen arbitrarily over the network in (33), we see that 

P^A\M c ^j = 0. 

The proof is completed. 



(34) 
□ 
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4.2 Consensus Analysis 



In this subsection, we present the consensus analysis of the proof of Theorem 4.1 Let Xiu(k) 
represent the j'th coordinate of Xi{k). Denote 

h(k) = min Xj[ 7 i(£;), H(k) = max xn^ik). 
i=l, ...,n i=l,...,n ,ul 

The consensus proof will be built on the estimates of S(k) = H{k) — h(k), which is summa- 
rized in the following conclusion. 

Proposition 4.2 Algorithm |5p achieves a global consensus if Gk is SUSC. 
Proof. Since P(.M C ) > P(.4 C ) = 1 when Qk is SUSC, we only need to prove 



P lim S(k) = 0\M C ) = 1. 
Let {x^ (k)}^L be a sample sequence in M. c . Then V£ = 1, 2, . . . , 3Ti(£, oj) > such that 

k>T x => <i i = l,...,n. (35) 



Moreover, based on similar analysis as in the proof of Lemma 4.2 we see that 



h{k + s)>h{k)-s-j, H(k + s)<H(k) + s-j (36) 

for all k>T\ and s > 0. 

Denote fei = Ti. Take vq G V with x^^i (&i) = h(k\). Then we can obtain from the definition 
of |[5j) that 

J ^ o [ ] (A;i) + i, if projection happens 

^ ,t#i + 1 ) ^ s ' ( 37 ) 

[ a^^i)^^]^!) + (1 - a UlVQ (kx))H(kx), if averaging happens 
which leads to that almost surely we have 

x V0;b] (h + 1) < V h(ki) + (1 - v)H(k 1 ) + j. 

Continuing the estimates we know that a.s. for any r = 0, 1, . . . , 

x UoM (h +r)< rfhikx) + (1 - rf)H{k x ) + T ±±A . (38) 

Furthermore, since Qy. is SUSC, we have 

p(3fci € + 1] and 3z/i G V s.t. (i/ ,fi) G > <?. 
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Let k\ = ki + g, < g < B — 1. Similarly with (24), we see from (38) that 



P(x uub] (h + Q+1)< V e +1 h(h) + (1 - n e+1 )H{kl) + V ■ • \\M C ) > pq. (39) 



2 

Similar analysis will lead to 

P(x vub] (k 1+ g+f) < v ^h(k l ) + (l-r l ^)H(k 1 )+ (g + T)(g 2 + T + 1} i, f = 1, 2, . . . |A^ C ) > pg, 

(40) 

which yields 

P^y^+B+r) < 7 7 g +^(fc 1 )+(l-^)g(fc 1 )+ (^±lX|+l+l) , 1 ? T = Q) 1; ^ j > pq 

We can continue the upper process on time intervals [k\ + 2B, k\ + 3B — 1], . . . , [ki + (rt — 
2)i?, fci + (n — l)i? — 1], and f2, . . . , v n -\ can be found until 
P( xvub] ( kl + (n - l)B) < V ^h( kl ) + (1 - n^Hfa) + (--DB((n-l)B + l) _ ^ 



2 



Z = 0,l,...,n-1|A4 C ) tj^- 1 ^- 1 . 



Therefore, denoting k 2 = fci + (n — 1).B, we have 



P(#(*2) < rj( n ~ l "> B h(ki) + (1 - 7 ? ("- 1 ) B ) J ff(fc 1 ) + ^ 1)i?((n 2 + • l\M c ) > p n ^q 



„n— 1 „n— 1 



Furthermore, with (36), we can further obtain 



P(s(k 2 ) < (1 - r ? ("- 1 ) B )5(A: 1 ) + L • > p"" 1 ?"- 1 , 

where L() = (n-DB^n-DB+Bl ^ 

Then we know Pjlim^oo S(k) = 0\Ai c } = 1 by similar analysis as the proof of Proposition 



4.1. The proof is completed. □ 



Theorem |4.1| immediately follows from Propositions 4.1 and 4.2 



5 Bidirectional Graphs 

In this section, we discuss the randomized optimal consensus problem under more restrictive 
communication assumptions, that is, bidirectional communications. To get the main result, we 
also need the following assumption in addition to the standing assumptions A1-A4. 
A5 (Compactness) Xq is compact. 

Then we propose the main result on optimal consensus for the bidirectional case. It turns 
out that with bidirectional communications, the connectivity condition to ensure an optimal 
consensus is weaker. 
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Theorem 5.1 Suppose Qk is bidirectional for all k > and A5 holds. Algorithm |5p achieves 
a global optimal consensus almost surely if Qk is SIC. 

Remark 5.1 The essential difference between SUSC and SIC graphs is that SIC graphs do not 
impose an upper bound for the length of intervals where the joint graphs are taken. Therefore, 
the analysis on directed graphs cannot be used in this bidirectional case. 

In the following two subsections, we will focus on the optimal solution set convergence and 



the consensus analysis, respectively, by which we will reach a complete proof for Theorem 5.1 



5.1 Set Convergence 

In this subsection, we discuss the convergence to the optimal solution set. First we give the 
following lemma. 

Lemma 5.1 Assume that is bidirectional for all k>0. Then P(^4n Ai) =0 if Gk is SIC. 



Proof. The proof follows the same line as the proof of Lemma 4.1 Let k m and T are defined 
the same way as the proof of Lemma 4.1 Suppose k m > T. Based on the definition of ([5]), we 
know from Lemma I3JJ that 



p(k ;i) (/>•„, + i>k, i , i ■ ]r 2 l fi) > i i>. (4i) 



Next, we define 



h = inf {3j G V s.t. (i ,j) G £ k }; Vi = {j £ V : (i ,j) € & ki }. 
Based on the definition of SIC graphs, we have for all r = 0, 1, . . . , 

P(3j G V,k € [k* T ,k* T+1 ) s.t. (i ,j) G £ k ) > p(g([k* T ,k* +1 )) is connected) > q. (42) 



Thus, Lemma 2.3 implies that the probability of k\ being finite is one. 



Applying Lemma 3.1 on node io, we have 

\xi (s)\ Xo <\xi (k m + l)\x , k m + 1 < s < ki. (43) 

As a result, we have 
PI 



+ l)k < »/y « + \) 2 - \sf o + (1 - ri)(!i + 1 e ),z€ Vi) > pl Vl l(l - p). (44) 
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We can repeat the upper process, V2, • • • , Vd can be denned iteratively for some constant 
1 < do < n — 1 until V \ {io} = Ufli V?'- Denoting ? m = kd + 1 associated with Vd , we have 



p( k,-u-,„ )i.v„ < AM + *) 2 - ^? + (1 - tf*>)(£ + € V 



= P( . max n |^MU < (f + - ^ + (1 - V d0 )(C + J) 
>p n-1 (l-p). (45) 
This will also lead to 

P(£*) = 1, (46) 



where ft„ = {f < r?™ -1 ^ 2 ~ + (1 - rf~ l )(}- Noting the fact that AC\M C 7^, the 
conclusion holds. □ 
Next, we define 

j/i = liminf |xi(/c)|x , i = 1, . . . ,n 
and denote P = {3io s.t. yi < £}. We give another lemma in the following. 
Lemma 5.2 Assume that Qj. is bidirectional for all k>0. Then P(^nD) =0 if Gk * s SIC. 



Proof. The proof will follow the same idea as the proof of Lemma 5.1 Let {x^ (fc)}^_ be a sample 
sequence. There exists a time sequence k\{uj) < ■ ■ ■ < k m {<jj) < . . . with lmim^oo k m {<jj) = 00 
such that 

K(Mw))|x <i(j«oM + eM). (47) 

Moreover, W = 1, 2, . . . , 3T(£, w) > such that 

fc>T => 0< |x?(fc)|x <£(") + 7, t = l,...,n. (48) 



Let fei and Vi follow the definition in the proof of Lemma 5.1, by the same argument as we 



obtain (44), we have 



P(V#l + 1)1*6 < Ivk + (1 - f )(* + 7).* e Vi) > p |Vl1 . (49) 
Continuing the upper process, we will also reach 

P( .maxJ^ m )U <^-y i0 + (l- I } ) " p ' (50) 
where 1 < do < n — 1 and q m still denotes kd + 1. Introducing 

W = {t<\.y l0 + {l-\).i), 
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we can similarly obtain P (W) = 1 according to ( 50 ) . The fact that A D T> C W c implies the 



desired conclusion immediately. □ 



Note that, if A5 holds, according to Lemma 3.2 for any initial condition x , we have 



Xi(k) G Xq, i = 1, . . . ,n; k = 0, 1, . . . , 

where Xq = {v : \v\x < d*} with = maxj = i v .. in |xj(0)|x - Then Xq is also a compact set, 
which is an invariant set for Therefore, for any initial condition, there will also be two 
constants b\, 62 > such that 



\xi(k) - xj(k)\ < 61; \xi(k)\ Xo < b 2 (51) 



for all i,j and k. 



Now we are ready to prove the optimal set convergence part of Theorem 4.1 which is stated 
in the following conclusion. 

Proposition 5.1 Assume Qk is bidirectional for all k > and A5 holds. Algorithm |5p achieves 
a global optimal set aggregation a.s. if Qt is SIC. 



Proof. With Lemmas 5.1 and 5.2, we only need to show 

P(AnM c nv c ) = 0. 

Take io £ V. Then we define two parallel hyperplanes 

W io (k) = {v\(x io (k) - P X() (x io (k)),v - x io (k)) =0} 

and 

W* Q (k) = {v\(x i0 (k) - P Xo (x to (k)),v- Px (x i0 (k))) = 0}. 

The space M, d is divided by the two hyperplanes into three disjoint parts M + {k) = {v\(xi (k) — 
P Xo (x io (k)),v - x io (k)) < 0}, M~(k) = {v\{x i() (k) - P Xo (x io (k)),v - P Xo (x io (k))) < 0}, and the 
rest M°(k) (see Figure |2j). Also define = {j\j is a neighbor of i for infinitely many k}. 

Claim. Pjlim^oo Ix^A;)!^^ =0,j G N™\An M c DV C } = 1. 

Let {x u) (k)} r ^ = Q be a sample sequence in An M C C\V C . Then W = 1,2, . . . , 3T(£, uj) > such 
that 

k > t < < kr(fc)u < e(w) + ^ and i<(*ok < ^ = i, . . . (52) 
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W i0 (k m ) 



WUK 



■0* 
t 



M+(fc m ) 



M-(fc m ) 



Figure 2: Finding the point x^ in the proof of Prop. 5.1 



Suppose there exist a constant $ w > and a sequence &i < • • • < k m < ... such that 
\ x j( k m)\w iQ (k m ) >^,m = l,2,.... Take fc m > T. With (jB, we see that for all k = 1, 2, . . . , 

*o C M-(fc) U Wj* (A;) = H^ioW - P Xo (x io (k)),v- P Xo (x io (k))) < 0}. 

Let x% (k m ) and Px ( x f (km)) be fixed. Then we can associate a unique point x^ to (fc m ) in 
the way that x^ satisfies (Px (xf (k m )) — x^ , xf o (k m )) — xj(k m )) = if the three points xf Q (k m ), 
Px (xf (k m )) and x^(k m ) form a triangle; and x* = Px (xf (k m )) otherwise. Moreover, it is not 
hard to find that there exists a unique scalar < 7 < 1 such that x^ = ^xf Q (k m ) + l) x< j (km)- 
Note that, the upper process defines a continuous function (xf o (k m ), Px (xf (k m )), x^(k m )) h-> 7. 



With (51), we have (xf Q (k m ),Px (xf (k m )),x l j(k m )) always locates within a compact set {0 < 
K(k m )\ Xo < d*; <T < \xf(k m )) - xf Q (k m )\ < h; < \xj(k m )) - P Xo (x%(k m ))\ < b x + b 2 }. 
Therefore, there exist two constants < 7* < 7* < 1 (by a constant, we mean it does not 
depend on k m ) such that 7* < 7 < 7* (see Fig. [2]). 

Thus, every linear combination of xf Q (k m ) and x^(k m ) can be rewritten into a linear combi- 
nation of xf Q (k m ) and x^, and the lower bound of the weights is preserved. We also have 

Kko < K - Px (xl(k m ))\ < sm/3 (£M + \) < + j), (53) 



where /3 = <P(x^(k m ) - xf o (k m ), P Xo (xf o (k m )) - xf (j (k m )) and < 6, = y/l - ( J) 2 



< 1. There- 



fore, with (53), repeating the deduction used in the proofs of Lemmas 5.1 and 5.2, the claim can 



then be proved. 

Next, since Qk is SIC, i.e., the joint graph is connected with probability q > independently 
for infinite times, letting be the graph generated by neighbor sets A/^°, it is obvious that 
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M-(jfe) 

Figure 3: Locating v* in the proof of Prop. |5.1 



Goo is connected with probability 1. Therefore, the upper analysis can then be further carried 
out on Goo following iq's neighbors, ?o's neighbors' neighbors, and so on, until we finally reach 

lim \xj(k)\ w . (fc ) = 0; (54) 

with probability 1 for all j G V conditioned A n M. c n V c . Thus, by the definition of Wi and 
([!]), we have 

Pf lim \P W * (xj(k)) - P Xo (xj(k))\ = 0, j = 1, . . . , n\A n M c n P c ) = 1. (55) 
Denote T io (k) = co{P x , {x io (k)), P Xo {x\(k)), . . . , P Xo (x n (k))}. Then T io (k) C X io ,\/k > 0. 



7i m (k) can then be defined for m = 1, . . . ,n — lin the same way. Therefore, with (54), and 
according to the structure of Wi (k) and W^* (A;), with probability 1 conditioned A n .M c fl P c , 
there will be a point G flLo^mW ^ -^o for sufficiently large k such that t>* G M°(A;) (see 
Fig. §, i.e., 

p(3A; s.i. (x io (A;) - P Xo (x io (k)), v* - PxoK,^))) > 0|„4n.M c nP c ) = 1. 

This implies P(U n^nF) =0 because P((y - Px (y),v* - Px (y)) > 0) = for any 
y G ]R d and G Xo according to ([I]). The proof is completed. □ 

5.2 Consensus Analysis 

This subsection focuses on the consensus analysis of Theorem |5.1| 



We define a multi-projection function:Pj fc j fc _ 1 ...j 1 : R m — > Ut=i -^i with ii, . . . , i*. G {1, . . . , n}, 
fe > 1 by Pi k i k _ 1 ...i 1 {v) = Px^Px^ ■ ■ ■ Px h (y)- Define P (y) = y as the case for fc = 0. Let 

r = {Pi k i k _ l ...i 1 ■■ h,...,i k e {i,...,JV} ) fc = o,i,2,...} 

24 



be the set which contains all the multi-projection functions. Denote = co{x\{k\ . . . ,x n (k)} 
be the convex hull of all the nodes's state at step k, and define Ay fc by Ay fe = co{P(y) \y E 
Yk, P E r}. Then it is not hard to see that Ay fc is actually an invariant set along algorithm ^ 
for any k > 0, i.e., x%{s) E Ay fe for all i, k and s > k. 

We present another lemma establishing an important property of Ay fc . 

Lemma 5.3 For any y E Ay fc; we have \y\y k < 2max ?/g y fc |y|x - 



Proof. With Lemma 2.2, any y 6 Ay K has the following form 



d+1 
i=l 

where ^f=i A* = 1 with Aj > 0, pW £ T and 2; E Fjf, i = 1, . . . , d + 1. Then, by the 
non-expansiveness property d2h, we have that for any z E R rf and P E T, 



|P.Y (z) " = I^XoC*)) - P (^)l ^ \ P *o (*) - *l = l*kr 



This leads to 



d+1 d+1 d+1 d+1 

^A 4 P«(^)-^A^| < ^Ail^-Pxo^)! +^A l |p Xo (^)-P <i> (^) 
«=i j=i «=i i=i 



< 2 max \ z\x , 



which implies the conclusion because Ylf=i ^i z i ^ ^fc- ^ 
We can now present the consensus analysis. 

Proposition 5.2 Assume that Qk is bidirectional for all k > and A5 holds. Algorithm |5p 
achieves a global consensus a.s. if Qk is SIC. 

Proof. We only need to show P ( lim/ c _ s . 00 S(k) = 0\A C ) = 1. Let {x u (k)}'^ =0 be a sample 
sequence in A c . Then W = 1, 2, . . . , 3Tj(^, w) > such that 

k>T x ^ \x?{k)\x < 1 ,i = l,...,n. (56) 



As a consequence, Lemma 5.3 implies 



h(k + s)>h(k)-j; H{k + s)<H{k) + j (57) 



for all k>T\ and s > 0. 

Denote k\ = T\. Take uq E V with = h(k\). Define 



fcj = m£ {3jeV s.t. {v ,j)eS k }; V x = {j E V : {u ,j) E %J. 
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With (56), we have 



Xv ,\j]{h) < x UOtb] (h) + - = h(kx) + j. 



Thus, 



p^ 1)b] (fei + 1) < nKh) + (i - v)H(h) + 

for any v\ € Vi, which leads to 



A c ) >p 



Plx^ih + 1) < r}h{k x ) + (1 - v)H(k 1 ) + - i € Vi 



(58) 
(59) 

(60) 



Similar with the proof of Lemma 5.1 we can repeat the upper process, and V2, • • • , Vd can 
be denned for some constant 1 < do < n — 1 until V \ {^0} = Ujli Vj- Moreover, we can also 
obtain that 



P[x i:b] (k do + l) < ^hih) + (l - v^Hih) + ieV 



2d 



A c ) >p 



n-l 



(61) 



Therefore, denoting &2 = kd + 1, we have 



P{H(k 2 ) <r 1 do h(k 1 ) + (l-rj d0 )H(k 1 ) + ^\A C ) >p 



2dn 



We see from (57) and (62) that 



P{S(k 2 ) < (1 - ^Sih) + L • 2{d \ - 1} |^ c ) > p n ~ l . 



Then we know P(limfc_>.oo S(k) = 0|^4 C ) = 1 by similar deduction as the proof of Prop. 
The proof is completed. 



4.1 



□ 



Then we see that Theorem 5.1 follows from Propositions 5.1 and 5.2 



6 Numerical Example 

In this section, we study a numerical example to compare the convergence rates of deterministic 
and randomized algorithms, and to illustrate the optimal choice of the decision probability p in 
the randomized algorithm. 

Consider a network with three nodes V = {1,2,3}. The communication graph is fixed and 
directed. Here £ = {(1, 2), (2, 3), (3, 1)} is the arc set. We take dij(k) = 0.5 for all (i,j) G £■ 
The optimal solution sets corresponding to the nodes are three disks in B 2 with radius 1 and 
centers (—1,0), (1,0) and (0,-1), respectively. Their intersection Xq = {(0,0)} is a singleton. 
Initial values for each node are (—2, 2), (—2, —2) and (2, —2), respectively. 
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We compare the randomized algorithm presented in this paper and the projected consensus 
algorithm in |41| . which is a deterministic algorithm with each node taking averaging and pro- 
jection alternatively. Numerical experiments show that the deterministic algorithm leads to a 
faster convergence than the mean performance of the randomized algorithm. The reason for this 
is natural since consecutive projections may take place for some nodes. However, surprisingly 
enough there are still about 5 percents of the experiments for which the randomized algorithm 
performs better than the deterministic one. Moreover, we also find that the randomized algo- 
rithm usually converge faster near Xq. See Figure |4| 
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Figure 4: The left figure shows the average performance of the randomized algorithm, and in 
the right one we select a typical sample when the randomized algorithm converges faster. Here 
D (k) = maxj = i j2 ,3 \xi(k)\ Xo - 



We further compare the average performance of the randomized algorithm when p takes 
values from 0.2, 0.5 and 0.8. Experiments show that the case when p = 0.5 reaches the fastest 
convergence. This is to say, it is better for the nodes to balance computation (projection) and 
communication (averaging) . See Figure [5j 
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10 20 30 40 50 

k 

Figure 5: Convergence rates for different decision probabilities (Do(k) = maxj = i 5 2,3 \ xi(k)\x )- 

7 Conclusions 

The paper investigated a randomized optimal consensus problem for multi-agent systems with 
stochastically time-varying interconnection topology. In this formulation, the decision process 
for each agent was a simple Bernoulli trial between following its neighbors or sticking to its 
own opinion at each time step. In terms of the optimization problem, each agent independently 
chose either taking an average among its time- varying neighbor set, or projecting onto the 
optimal solution set of its own objective function randomly with a fixed probability. Both 
directed and bidirectional communications were studied, and stochastically jointly connectivity 
conditions were proposed to guarantee an optimal consensus almost surely. The results showed 
that under this randomized decision making protocol, a group of autonomous agents can reach an 
optimal opinion with probability 1 with proper convex and nonempty intersection assumptions 
for the considered optimization problem. Fundamental challenges still lie in the convergence 
rate of the randomized algorithm and the choice of optimal decision probability to reach a faster 
convergence. 
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